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PREFACE 


This  paper  gives  an  overview  of  R-WISE  (Reading  and  Writing  in  a  Supportive  Environment). 
This  software  is  an  adaptive  tutoring  environment  that  fosters  (1)  verbal  reasoning  skills,  (2) 
strategy  acquisition  for  composition,  and  (3)  the  metacognitive  awareness  necessary  to 
manage  the  multi-dimensional  nature  of  the  wnting  process. 

Many  people  contributed  to  the  development  of  R-WISE.  The  authors  express  their  gratitude 
to  the  high  school  teachers  who  served  as  subject  matter  experts;  the  programmers  who 
implemented  the  software;  and  the  research  assistants  who  tabulated  the  data  for  the  pilot 
study.  We  especially  acknowledge  the  generous  sharing  of  time  and  talent  and  continued 
support  of  the  following  individuals:  Dr.  Wes  Regian,  Senior  Scientist  for  the  Intelligent 
Training  Branch  of  the  Armstrong  Laboratory  (AL/HRTI);  Lt.Col.  James  Parlett,  AL/HRTI 
Branch  Chief;  Dr.  Kurt  Steuck,  the  FST  Project  Manager;  and  Ms.  Teri  Jackson,  who 
oversaw  the  implementation  of  R-WISE  at  ten  different  sites. 


R-WISE:  A  LEARNING  ENVIRONMENT 
TO  TEACH  PROSE  COMPOSITION 


1.0  BACKGROUND  AND  PURPOSE 

1.1  The  Fundamental  SkMIs  Training  Project  (FST) 

The  Fundamental  Skills  Training  Project  (FST)  is  a  unique  and  timely  collaboration  among 
the  military,  business/industry,  and  the  educational  community  to  address  one  of  the  most 
pressing  challenges  facing  our  nation;  teaching  the  fundamental  skills  necessary  to  participate 
in  a  complex,  modem  society.  Of  the  initial  set  of  three  tutors,  one  (an  algebraic  word 
problem  solving  intelligent  tutor)  has  been  completed;  a  second  (a  critical  literacy  tutor)  is 
in  the  evaluation  phase;  and  a  third  (a  principles  of  science  tutor)  is  in  the  development 
stage.  This  document  elaborates  on  the  critical  literacy  tutor. 

1.2  Critical  Thinking  and  Composition 

The  national  attention  focused  on  the  "Uteracy  crisis"  addresses  a  significant  and  real  problem 
in  contemporary  America  -  the  alarming  increase  in  numbers  of  people  who  simply  cannot 
read.  However,  this  high  visibility  for  illiteracy  in  its  most  egregious  forms  may  mask  other 
issues  in  "language  skills."  For  example,  recent  National  Assessment  of  Educational  Progress 
(NAEP)  findings  suggest  a  growing  inability  among  young  people  to  perform  complex  verbal 
tasks,  to  draw  inferences  from  text,  to  detect  bias  in  verbal  presentations,  to  follow  multi-level 
arguments.  In  other  words,  NAEP  identified  a  substantial  stratum  of  learners  who  can  read 
and  write  at  a  level  of  minimal  competency  (and  are  therefore  not  "functional  illiterates  ) 
but  who  cannot  demonstrate  deep  understanding,  cannot  use  probabihstic  reasoning  or  draw 
valid  conclusions,  cannot  distinguish  among  levels  of  specificity  or  forms  of  evidence,  cannot 
follow  implications  in  text,  and  in  general  are  prone  to  oversimplify  all  but  the  most  basic  of 
verbal  tasks. 

Deficiencies  in  composition  are  especially  noteworthy.  Most  adolescent  learners  write  only  on 
a  minimal  level  of  acceptability.  Furthermore,  their  inadequacy  is  not  necessarily  a  result  of 
poor  spelling,  vocabulary,  grammar,  verbal  fluency,  syntax,  paragraphing,  or  other  such 
production  skills.  Bereiter  and  Scardamalia  call  the  form  of  writing  practiced  by 
inexperienced  writers  knowledge  telling  (1987,  p.  5).  For  these  two  noted  researchers, 
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knowledge  telling  is  characterized  by  (1)  a  simple  task  execution  involving  only  limited 
planning  and  minimal  mental  engagement,  (2)  production  methods  adapted  from  oral  abilities, 
(3)  organizational  patterns  based  on  free-association  or  simple  narration,  (4)  development  that 
contains  large  chunks  of  irrelevant  information  or  elaborations  based  on  simple  descriptions, 
and  (5)  an  egocentric  perspective.  The  antithesis,  knowledge  transformation,  as  practiced 
by  good  writers,  is  characterized  by  (1)  guided  planning  and  situational  diagnostics,  (2)  rich 
mental  representations  of  text  possibilities  for  a  wide  range  of  scenarios,  and  (3)  a  robust 
"executive  control  program"  for  allocating  mental  resources  and  for  handling  the  tremendous 
cognitive  load  of  verbal  composition. 

R-WISE  addresses  these  issues  of  critical  literacy  and  teaches  the  use  of  language  as  a  vehicle 
for  critical  thinking.  Though  we  draw  our  philosophy  as  much  from  informal  logic  (on  the 
rhetoric  side)  and  problem-solving  (on  the  cognitive  science  side),  we  also  draw  (particularly 

in  the  area  of  pedagogy)  from  the  whole-language  movement.  The  goal  of  the  project  is  to 

design,  develop,  field,  and  evaluate  an  intelligent  tutoring  system  to  teach  critical  thinking 
skills,  as  manifested  in  reading  and  wnting. 


2.0  COMPUTERS  AND  THE  TEACHING  OF  WRITING 

2.1  Historical  Approaches 

The  notion  of  using  computers  to  facilitate  the  teaching  of  writing  has  been  around  for  quite 
some  time  (Bangert-Drowns,  1993).  In  general,  one  can  see  two  broad  trends  in  the  early 
efforts;  (1)  research  into  whether  or  not  word  processing  alters  composing,  and  (2)  various 
forms  of  parsers  and  natural  language  processing  for  detecting  flaws  in  text  (such  as  the  many 
separate  programs  in  the  Writer's  Workbench®).  Both  efforts  have  produced  only  limited 
results  because  both  technologies  -  at  their  roots  ~  are  intended  to  enhance  productivity 
rather  than  facilitate  cognition.  In  short,  they  are  just  not  the  type  of  tool  -  as  delmeated  by 
Gavriel  Salomon  (1991  and  1993)  -  to  produce  a  lasting  gain  in  performance,  once  the  tool 

has  been  removed. 

2.2  R-WISE  as  a  Learning  Environment 

R-WISE  encourages  students  to  practice  composition  in  a  computer-mediated  environment, 
where  the  specially  designed  software  acts  as  a  procedural  facilitator.  This  term  is  used  by 
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Vygotsky  (1978)  to  explain  the  cognitive  mentoring  and  developmental  dynamics  that 
occur  between  master  and  apprentice  and  between  peers  during  collaboration.  Salomon 
(1988)  and  Zellermayer  (1991),  among  others,  use  the  term  to  suggest  that  the  computer  can 
serve  asn  peripheral  brain  for  the  fledgling  student  and  provide  the  scaffolding  that  allows  the 
novice  to  practice  the  more  robust  problem-solving  behaviors  of  an  expert.  R-WISE 
serves  as  a  cognition  facilitator  for  critical  thinking  by: 

•  Easing  demands  on  short-term  memory  and  helping  to  focus  attention  on  strategically 
important  aspects  of  writing; 

•  Guiding  the  inculcation  and  self-initiation  of  higher-order  processes  (metacognition)  which 
the  novice  writer  is  unlikely  to  activate  without  prompting; 

•  Explicitly  modeling  strategic  inteUectual  processes  so  that  the  fledgling  student  avoids 
what  Collins  &  Centner  (1980)  have  termed  "downsliding,"  or  becoming  increasingly 
entangled  in  lower  and  lower  levels  of  mental  actions,  finally  concentrating  all  mental 
energy  on  such  things  as  spelling,  grammar,  and  sentence  construction  to  the  exclusion 
of  larger  concerns  in  the  process. 


3.0  SOFTWARE  COMPONENTS 

R-WISE  consists  of  a  suite  of  computerized  "tools"  to  aid  ninth-graders  in  learning  the  art  of 
prose  composition.  We  selected  the  tools  based  on  (1)  their  potential  to  represent 
components  of  the  writing  process  as  a  visual  algorithm  and  (2)  their  ability  to  model  robust, 
expert  behaviors  for  the  student.  A  simple  model  for  the  composing  process  that  has  gained 
wide  acceptability  in  pedagogy  depicts  writing  as  having  three  central  phases  or  stages: 
(1)  Prewriting  or  invention,  (2)  Writing  or  drafting,  and  (3)  Revision  or  editing.  The 
three  software  components  showcased  in  this  paper  mirror  this  partitioning  of  the  process. 

3.1  Cubing;  The  Prewritin2  Tool 

Information  is  not  knowledge,  just  as  understanding  the  content  of  a  piece  of  prose  is  not 
equivalent  to  understanding  and  using  the  concepts  presented  in  a  text.  This  tool  is  called 
"Cubing"  because  it  is  based  on  a  graphical  representation  of  complex  mental  operations. 
"Cubing"  helps  the  student  to  draw  inferences  and  to  elaborate  on  the  basic  concepts  being 
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developed  in  a  piece  of  emergent  text.  The  tool  helps  to  bootstrap  comprehension  by  model¬ 
ing  a  process  for  (1)  extracting  meaning,  (2)  formulating  inferences,  interpretations,  and  re¬ 
interpretations,  and  (3)  engaging  passive  knowledge  and  connecting  it  with  the  current  stimuli. 

3.2  Idea  Board:  The  Draftm2  Tool 

This  tool  (called  Idea  Board)  mediates  a  major  cognitive  shift  in  the  writing  process  ~  from 
the  macro  structures  of  thought  to  the  micro  structures  of  socially  accepted,  connected  prose. 
Using  a  visual  algorithm,  the  tool  models  robust,  expert  behaviors  for  writing  a  first  draft. 
By  foregrounding  activities  at  appropriate  times  and  relegating  others  to  less  prominence,  the 
tool  teaches  the  student  to  manage  the  cognitive  load  of  composition. 

3.3  Triple  Vision:  The  Revision  Tool 

This  tool  helps  the  student  to  "re-see"  a  completed  draft  at  three  different  levels  of  focus. 
Editing,  as  fostered  in  this  tool,  refers  to  substantial  changes,  such  as  improving  style,  adding 
to  or  subtracting  from  the  content,  rearranging  parts,  or  completely  rewriting.  These  more 
global,  deep-structured  editing  acts  are  associated  with  higher-order  cognitive  skills  (e.g., 
discerning  patterns  in  bodies  of  information,  exercising  judgment,  analysis,  synthesis).  More 
specifically.  Triple  Vision  contains  a  number  of  heuristics  for  improving  both  coherence  and 
development. 


4.0  INSTRUCTIONAL  APPROACH 

R-WISE  uses  a  hybrid  paradigm  for  interactive  instruction.  Part  of  the  guidance  comes  from 
adaptive  tutoring  using  traditional  AI  formalisms  and  part  of  the  teaching  comes  from  the 
powers  of  reification  (or  representing  complex  processes  as  manipulable  objects  on  the 
computer  screen).  Tools  in  this  suite  (1)  accommodate  deficiencies  and  thereby  reduce 
frustration  for  a  weak  writer,  (2)  emulate  some  of  the  crucial  functionality  of  paper  copy, 
(3)  enrich  the  environment  and  thereby  sustain  motivation,  and  (4)  model  robust  behaviors. 

While  each  of  the  three  adaptive  tools  being  considered  concentrates  on  a  specific  cluster  of 
skills,  all  three  have  a  unified  method  for  delivering  layered  instruction  and  a  canomcal 
architecture  for  the  software  and  the  interfaces.  Figure  1  guides  the  discussion  for  the  next 
five  subheadings. 
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Figure  1:  System  Overview  of  Hybrid  Tutoring  Capabilities 
4.1  Setting  Goals 

(Area  1  of  Figure  1)  Using  a  knowledge-telling  approach,  the  novice  writer  views 
composition  as  if  it  were  a  straightforward  exercise  in  generating  ideas  through  association. 
For  the  expert,  however,  having  an  explicit,  stable  set  of  goals  fosters  a  kind  of  filtering 
activity  that  focuses  the  task  fi'om  the  outset. 

Each  of  the  three  cognitive  tools  handles  this  concretizing  of  goals  in  a  slightly  different 
manner.  In  Cubing,  the  tool  selects  from  a  collection  of  nine  possible  readers  and  nine 
possible  reasons  to  write,  (e.g.,  write  a  persuasive  piece  for  a  reader  who  is  an  expert  in  the 
subject  matter).  The  student  is  then  given  this  combination  as  a  rhetorical  situation.  Idea 
Board  asks  the  student  to  pick  from  a  list  the  most  applicable  response  to  each  of  four 
questions;  (1)  What  is  your  purpose  for  writing?  (2)  How  will  your  audience  react?  (3) 
What  is  the  best  form  for  this  paper?  (4)  What  difficulties  will  you  have  with  this  paper? 
Triple  Vision  asks  the  student  to  set  "sliders"  to  indicate  three  dimensions  of  the  reader  and 
three  dimensions  of  the  writer  (as  illustrated  in  Figure  2). 

Not  only  does  this  exercise  help  the  novice  student  focus  on  an  area  where  she  is  weak,  this 
preliminary  work  "sets"  the  parameters  of  the  adaptive  tutor.  Each  writing  environment  now 


5 


has  a  "frame"  or  backplane  of  conditions  against  which  further  actions  can  be  evaluated  during 
the  remainder  of  the  session  on  the  tool.  (If  the  student  changes  goals,  the  frame  is  also 
updated.)  Table  1  gives  the  number  of  combinations  (or  rhetorical  situations)  tracked  by 
each  of  the  tools.  Clearly,  the  repertoire  is  rich,  and  becomes  even  richer  as  these  preliminary 
combinations  are  conjoined  with  additional  data  points  drawn  from  the  student  s  subsequent 
activities  as  the  writing  progresses  (as  will  be  illustrated  in  Table  2). 


Goal  Specifications 


fteader*s  Age 


Reader's  Knowledge 


Reader's  Attitude 


Writer’s  Distance 


Writer's  Tone 


Writer's  Purpose 


How  we  "sound"  when  we  write  depends  a  great  deal  on 
the  age  of  the  reader.  For  example,  you  would  not  write 
the  same  way  for  your  high  school  principal  as  you  would 
for  your  seven-year-old  brother. 

Using  the  mouse,  move  the  slider  to  the  position  that  best 
represents  your  reader's  age  in  relation  to  your  own. 


Go  On 


Figure  2:  Setting  Parameters  for  System  Adaptation 
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Table  1:  Tracking  Combinations  in  the  Frame 


TOOL 

CONDITIONS 

COMBINATIONS 

Cubing 

Reader  Profiles  (9) 

Aims  or  Purposes  (9) 

81 

Idea  Board 

Purpose  (5) 

Audience  (6) 

Form  (6) 

DifiBculties  (7) 

1,260 

Revision 

Reader's  Age  (3) 

Reader's  Knowledge  (3) 
Reader's  Attitude  (3) 
Writer's  Distance  (3) 
Writer's  Tone  (3) 

Writer's  Purpose  (3) 

729 

4.2  Visual  Algorithms 

(Area  2  of  Figure  1)  The  second  way  in  which  R-WISE  teaches  is  to  reify  complex  and 
potentially  covert  mental  operations.  Where  possible,  the  interfaces  of  R-WISE  represent 
visual  organizers  for  specific  intellectual  processes.  As  explained  by  J.  H.  Clarke  (1991,  p. 
526-7),  "[f]rom  the  standpoint  of  cognitive  theory,  graphic  fi-ames  mimic  aspects  of  semantic 
memory  structures  or  schemata,  that  learning  theorists  believe  organize  the  mind. 

For  example.  Figure  3  shows  "Idea  Map,"  an  early  workspace  in  Idea  Board  that  encourages 
structured  brainstorming  by  presenting  the  student  with  a  visualization  of  mental 
manipulations.  Given  the  premise  that  most  of  the  clients  for  R-WISE  probably  have  leammg 
preferences  that  are  concrete/visual  rather  than  abstract/language,  we  provide  "objects"  for 
obscure  mental  actions.  Similar  to  "webbing"  ~  a  paper-and-pencil  techmque  used  in  the 
writing  classroom  —  this  thinking  frame  prods  the  student  to  cluster  ideas  into  proto¬ 
paragraphs.  Working  with  the  "Idea  Map"  helps  the  student  to  recognize  and  to  take  control 
of  the  intellectual  processes  foundational  to  composing. 
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Figure  3;  Idea  Map  Interface  from  the  Idea  Board  Tool 


4.3  Diagnosis  and  Repair 

(Area  3  of  Figure  1)  Writing  is  a  complex,  multi-dimensional  activity  analogous  to  a  contin¬ 
gency  management  problem.  Only  in  working  through  candidate  solutions  does  the  nature  of 
the  problem  become  fixed,  or  even  definable.  Rather  than  working  in  a  Unear  fashion  through 
the  model  of  PLAN  -  PREWRITE  -  DRAFT  -  EDIT,  good  writers  use  an  opportunistic 
approach.  They  constantly  measure  the  emerging  text  against  a  set  of  expectations,  while  at 
the  same  time  recognizing  and  capitalizing  on  serendipitous  gains,  weaving  these  "discovered" 
possibiUties  into  a  new  rendition  of  the  overall  plan  and  product. 

Unfortunately,  text  production  strategies  for  novice  ■writers  are  frail  and  one-dimensional. 
Such  impoverished  capabilities  do  not  lend  themselves  to  interruptions  or  re-assessments.  As 
evidenced  by  Bereiter  and  Scardamalia's  research  into  the  writing  process  for  novices,  little 
evidence  can  be  found  that  weak  writers  can  participate  in  self-cueing  or  self-monitonng 


8 


activities  while  engaged  in  the  production  of  text  (1987).  In  fact,  the  very  act  of  breaking  out 
of  their  one-dimensional,  stream-of-consciousness  mode  jeopardizes  the  continued  production 
of  text. 


Panel  1 


What  will  your  reader  think 
about  this  paragraph? 

*  Does  not  have  a  focus 
or  purpose 

*  Drifts  from  the  topic 
or  the  purpose 

*  Does  not  have  enough 
depth 

*  Airangement  of  ideas  is 
not  effective 

*  This  is  a  good  paragraph 


Figure  4:  Diagnosis  and  Repair  Prompting  for  Triple  Vision 


IF:  The  paragraph  does  not  have 

a  focus  or  purpose 

THEN: 

1  Identify  the  central  idea  of 
the  paragraph 

2  Make  sure  a  topic  sentence 
introduces  the  central  idea 

3  Relate  all  sentences  in  the 
paragraph  to  this  main  idea 

4  Order  the  sentences  in  a 
logical  progression  that 
reflects  your  purpose 


Panel  2 


Diagnosis-and-Repair  is  an  evaluation  loop  that  partners  with  the  student  to  reduce  the 
cognitive  load  and  encourages  the  student  to  enter  into  an  assessment  episode.  This  loop  - 
essentially  modeled  after  Bereiter  and  Scardamalia's  well-researched  CDO  (COMPARE, 
DIAGNOSE,  OPERATE)  sequence  (1987)  -  takes  a  very  sophisticated,  open-ended  problem 
and  pares  it  down  to  a  manageable  set  of  options  for  the  inexperienced  writer. 


Succinctly,  this  facilitator  operates  (in  all  three  tutors)  in  the  following  sequence: 

•  The  student  detects  a  mismatch  between  the  goals  (her  intentions  for  writing)  and  a  chunk 
of  text.  As  a  response  to  this  dissonance,  the  student  requests  help. 

•  The  system  brings  up  a  list  of  potential  problems  applicable  to  the  specific  sub-task  the 
writer  is  working  on.  For  example,  in  Figure  4,  the  student  is  reviewing  paragraph  patterns 
in  the  Triple  Vision  tool  and  detects  an  awkward  place. 
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•  Asking  for  help  produces  the  Diagnostics  on  Panel  1  (Figure  4).  Selecting  any  one  of  the 
first  four  assessments  brings  up  a  repair  agenda  (Panel  2)  dealing  specifically  with  that 
condition.  By  presenting  a  limited  set  of  options  and  by  making  suggestions  (rather  than 
dictating)  about  ways  to  improve,  the  system  both  engages  and  challenges  the  writer  at  the 
appropriate  level. 

Clearly,  the  lists  of  options  are  a  form  of  embedded  instruction,  and  the  student  probably  will 
come  away  from  extended  exposure  to  any  of  the  three  tutors  with  better  content  knowledge 
about  what  can  go  wrong  at  various  stages  of  composition.  However,  we  feel  that  the  more 
important  lesson  the  student  learns  is  an  enriched  self-regulatory  capacity  so  that  she  can 
move  out  of  the  text  production  mode  into  a  higher-level  cognitive  activity  without  disrupting 
the  whole  composing  process.  This  ability  to  suspend  operations  on  one  level  and  to  focus 
mental  energies  on  another  is  characteristic  of  the  experienced  writer  (Hayes  and  Flower, 

1980). 

4.4  Adaptive  Advice 

(Area  4  in  Figure  1)  In  the  metacognitive  stage  (diagnosis  and  repair),  the  machine  partners 
with  the  student  to  develop  the  sensitivity  and  awareness  necessaiy  to  know  what  is  wrong 
with  a  prose  performance  and  how  to  improve  the  result.  Yet,  because  the  diagnostic  is  per¬ 
formed  by  the  student,  there  is  a  potential  for  a  misjudgment.  Additionally,  if  the  system  is  to 
serve  as  an  intelligent  "guide"  or  "coach,"  the  tool  should  have  a  feedback  loop  to  indicate  the 
"reasonableness"  of  the  course  of  action  the  student  is  pursuing,  baselined  against  some 
known  set  of  criteria. 

Adaptive  advice  adjusts  its  statements  based  on  an  "intelligent"  assessment  of  the  situation  - 
meaning  that  the  software  compares  the  manipulation  the  student  is  working  on  with  the 
conditions  of  the  frame  and  determines  how  "correct"  these  actions  are  given  the  circum¬ 
stances.  The  resultant,  targeted  prompts  help  the  student  to  learn  the  more  subtle  aspects  of 
adaptation  to  audience  and  purpose.  They  also  help  the  student  to  stay  on  the  right  track  and 
avoid  the  fhistration  of  writing  text  that  is  later  deemed  to  be  inadequate  to  the  task.  Cubing, 
Idea  Board,  and  Triple  Vision  contain  advice  that  is  germane  to  the  focus  of  the  tool.  For 
example.  Cubing  —  whose  domain  is  prewriting  or  the  invention  phase  of  writing  —  prods  the 
student  to  generate  ideas.  Triple  Vision,  on  the  other  hand,  contains  advice  to  aid  in  the 
assessment  of  such  things  as  coherence,  introductory  and  concluding  paragraphs,  whole- 
paper  arrangement,  paragraph  structures,  and  effectiveness  of  individual  sentences.  Figure  5 
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shows  how  adaptive  advice  is  displayed  to  the  student.  In  this  particular  case,  the  student  is 
working  in  Cubing  and  is  generating  ideas  to  include  in  a  job  application.  The  reader  is 
classified  as  an  "evaluator,"  and  the  aim  is  "to  persuade."  The  student  is  trying  to  elaborate  on 
the  topic  uf  "previous  work  experience"  when  she  asks  for  strategic  advice. 


ADVICE  FOR  CUBING 


D  Ask  yourself,  "What  is  the  most  important 

feature  or  the  dominant  impression  given  by 
previous  work  experience?"  Because  your 
reader  knows  more  about  this  topic  than  you 
PfBScy^Uotu  push  yourself  to  come  up  with  something 
that  is  not  stale  and  obvious. 


Figure  5:  Example  of  Situation-Specific  Adaptive  Advice  for  Cubing 


In  all  the  intelligent  tools,  adaptive  help  is  generated  through  a  kind  of  triangulation,  based  on 
the  rhetorical  situation  (fi'ame  conditions  of  audience  and  purpose)  and  the  moves  made  by  the 
student  in  the  micro-world  or  visual  workspace.  Monitoring  the  combination  of  rhetorical 
situation  and  place  in  the  writing  process  creates  a  rich  representation  for  accessing 
instructional  statements.  Table  2  shows  the  number  of  instructional  situations  captured  in 
each  tool.  Because  of  the  large  numbers,  in  the  current  version  of  R-WISE  we  have 
implemented  a  pruning  algorithm  that  weighs  the  various  elements  going  into  the  instructional 
situation  and  generates  a  manageable  set  of  instructional  statements.  For  example,  the 
student,  having  written  a  topic  sentence  (1)  for  a  paragraph  of  factual  detail,  (2)  intended  for 
an  audience  at  a  distance  fi'om  the  writer,  (3)  younger  than  the  writer,  and  (4)  less 
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knowledgeable  than  the  writer  might  be  prompted  to  consider  whether  or  not  the  advanced 
organizer  (topic  sentence)  is  adequate  for  the  purpose  and  the  audience. 


Table  2;  Generating  Adaptive  Help 


TOOL 

FRAME 

COMBINATIONS 

DIAGNOSTIC 

CHOICES 

INSTRUCTIONAL 

SITUATIONS 

ADVICE 

STATEMENTS 

Cubing 

81 

18 

1,458 

72 

Idea  Board 

1260 

18 

22,680 

50 

Revision 

729 

27 

19,683 

153 

4.5  Just-in-Tinie-Tutorin2 

(Area  5  in  Figure  1)  While  designing  R-WISE,  we  carefully  planned  how  to  integrate  the  tutor 
into  a  year-long  ninth-grade  curriculum.  As  currently  fielded,  the  tutor  takes  up  about  20- 
25%  of  the  course.  The  production  skills  necessary  for  writing  (e.g.,  topic  sentences, 
paragraph  patterns,  conclusions  and  introductions,  and  other  rhetoric  fundamentals)  are  taught 
in  the  classroom,  not  on  the  computer.  This  is  a  deliberate  decision.  To  act  as  an  accelerator, 
the  computer  has  to  support  the  process  of  literacy.  Interrupting  the  process  to  teach  the 
enabling  skills  (1)  mixes  levels,  styles,  and  purposes  of  instruction,  (2)  creates  breaks  m  the 
train  of  thought  from  which  the  student  cannot  recover,  and  (3)  results  in  a  fairly  unexciting 

electronic  workbook. 

While  production  skills  and  metacognitive  skills  are  not  interchangeable,  they  are  correlated  in 
that  they  must  occur  simultaneously  in  expert  behaviors.  After  diagnosing  a  problem  and 
getting  a  repair,  the  student  may  still  be  at  a  loss  as  to  what  to  do.  Recognizing  that  students 
may  need  reminders  of  materials  covered  in  class,  we  have  embedded  a  CBT  component  in 
R-WISE  that  provides  "hints"  upon  request.  This  instruction  (similar  to  a  high-end  form  of 
context-sensitive  help)  is  analogous  to  a  job  aid  in  that  it  gives  a  synoptic  overview  of 
concepts  presented  in  class.  Its  purpose  is  to  serve  as  a  reminder  or  a  refocusing  prompt  for 
the  student  rather  than  a  full-blown  instructional  component. 
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All  three  tools  have  a  just-in-time-tutoring  (JITT)  module  for  each  diagnostic  choice  they 
present.  Each  unit  is  terse  and  highly  visual,  representing  fundamentals  of  composition  with 
conceptual  maps  and  pseudo-animation.  Figure  6  gives  a  sample  of  a  JITT  interface. 


mM 


CLUSTER  AND  SUBHEADING 


Die  Edit  lexl  Eage  Help 


Look  for  ideas  that  caii  be 
giouped  together. 


First,  look  at  the  content  of 
your  paragraph  as  a  "cluster" 
diagram. 

A  cluster  that  has  more  than  4 
or  5  spokes  off  your  main  idea 
can  be  improved  by  adding 
subheading  ideas. 

This  type  of  paragraph  'will 
sound  like  a  long  LIST  'when 
you  read  your  paragraph  again. 


Cluster  diagrams  help  you  sort  your  ideas  into  logical 
groups. 


Figure  6;  Example  of  a  Single  Screen  from  a  JITT  Module 


5.0  PILOT  STUDY  AND  FUTURE  PLANS 

A  pilot  version  of  R-WISE  was  field-tested  during  academic  year  1992-1993.  A  San  Antonio 
high  school  with  a  population  of  approximately  650  ninth-graders  used  various  components  of 
the  system  over  a  nine-month  period.  In  working  with  this  Beta-test  site,  we  were  interested 
in  a  variety  of  research  questions,  including  issues  of  design  and  user  acceptance.  We  used 
several  instruments  to  measure  learning  outcome,  but  only  the  results  of  the  writing  sample 
are  presented  here. 
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A  second  high  school,  within  the  same  district  and  demographically  similar  to  our  pilot  site, 
was  selected  as  a  control.  Two  equivalent  writing  prompts  were  devised,  and  one  was  given 
in  January  as  a  pre-test;  the  other  was  given  in  May  as  a  post-test.  We  devised  a  rubric  for 
holistically  scoring  the  papers  on  a  1  to  6  scale  and  then  had  the  approximately  2,200  samples 
professionally  evaluated.  The  standard  procedure  of  having  two  readers  examine  each  paper 
was  used.  Inter-rater  reliability  was  .79  ~  meaning  that,  for  close  to  80%  of  the  writing 
samples,  a  team  of  two  readers  gave  the  paper  the  same  number  during  independent 
evaluation.  This  is  a  rigorous  standard  for  reliability.  The  more  common  practice  in  wnting 
sample  scoring  is  to  declare  continuous  numbers  (e.g.,  4  and  5  or  3  and  2)  reported  by  two 
readers  as  being  in  agreement.  Had  we  used  this  method  of  determining  agreement,  our  inter¬ 
rater  reliability  would  have  approached  .92. 


Figure  7:  Mean  Scores  for  Control  and  Treatment  on  Pre  and  Post  Test 

The  differences  in  means  on  the  pre-test  and  post-test  for  both  control  and  treatment  groups 
were  deemed  statistically  significant  using  t-tests  and  are  given  in  Figure  7.  We  are 
encouraged  by  the  approximately  8%  gain  accomplished  by  the  treatment  group,  but  are 
cautious  in  our  interpretations  given  that  the  software  was  in  a  Beta  version  during  the  pilot 
study.  A  more  finished  version  of  R-WISE  is  now  being  used  by  ten  test  sites  in  five  different 
states.  We  hope  to  get  even  more  of  an  effect  from  this  production  version  of  the  software. 
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